PAC-Bayesian Analysis of the Exploration-Exploitation Trade-off
نویسندگان
چکیده
We develop a coherent framework for integrative simultaneous analysis of the explorationexploitation and model order selection tradeoffs. We improve over our preceding results on the same subject (Seldin et al., 2011) by combining PAC-Bayesian analysis with Bernstein-type inequality for martingales. Such a combination is also of independent interest for studies of multiple simultaneously evolving martingales.
منابع مشابه
PAC-Bayes-Bernstein Inequality for Martingales and its Application to Multiarmed Bandits
We combine PAC-Bayesian analysis with a Bernstein-type inequality for martingales to obtain a result that makes it possible to control the concentration of multiple (possibly uncountably many) simultaneously evolving and interdependent martingales. We apply this result to derive a regret bound for the multiarmed bandit problem. Our result forms a basis for integrative simultaneous analysis of e...
متن کاملDopaminergic Control of the Exploration-Exploitation Trade-Off via the Basal Ganglia
We continuously face the dilemma of choosing between actions that gather new information or actions that exploit existing knowledge. This "exploration-exploitation" trade-off depends on the environment: stability favors exploiting knowledge to maximize gains; volatility favors exploring new options and discovering new outcomes. Here we set out to reconcile recent evidence for dopamine's involve...
متن کاملBoldness predicts an individual's position along an exploration–exploitation foraging trade‐off
Individuals do not have complete information about the environment and therefore they face a trade-off between gathering information (exploration) and gathering resources (exploitation). Studies have shown individual differences in components of this trade-off but how stable these strategies are in a population and the intrinsic drivers of these differences is not well understood. Top marine pr...
متن کاملA PAC-Bayesian Analysis of Graph Clustering and Pairwise Clustering
We formulate weighted graph clustering as a prediction problem1: given a subset of edge weights we analyze the ability of graph clustering to predict the remaining edge weights. This formulation enables practical and theoretical comparison of different approaches to graph clustering as well as comparison of graph clustering with other possible ways to model the graph. We adapt the PAC-Bayesian ...
متن کاملA PAC-Bayesian Analysis of Co-clustering, Graph Clustering, and Pairwise Clustering
We review briefly the PAC-Bayesian analysis of co-clustering (Seldin and Tishby, 2008, 2009, 2010), which provided generalization guarantees and regularization terms absent in the preceding formulations of this problem and achieved state-ofthe-art prediction results in MovieLens collaborative filtering task. Inspired by this analysis we formulate weighted graph clustering1 as a prediction probl...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- CoRR
دوره abs/1105.4585 شماره
صفحات -
تاریخ انتشار 2011